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Abstract 

The Lovasz Local Lemma due to Erdos and Lovasz (in Infinite and Finite Sets, Colloq. Math. Soc. 
J. Bolyai 11, 1975, pp. 609-627) is a powerful tool in proving the existence of rare events. We present 
an extension of this lemma, which works well when the event to be shown to exist is a conjunction of 
individual events, each of which asserts that a random variable does not deviate much from its mean. 
As applications, we consider two classes of NP-hard integer programs: minimax and covering integer 
programs. A key technique, randomized rounding of linear relaxations, was developed by Raghavan 
& Thompson {Combinatorica, 7 (1987), pp. 365-374) to derive good approximation algorithms for 
such problems. We use our extension of the Local Lemma to prove that randomized rounding produces, 
with non-zero probability, much better feasible solutions than known before, if the constraint matrices of 
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^ ' these integer programs are column-sparse (e.g., routing using short paths, problems on hypergraphs with 

small dimension/degree). This complements certain well-known results from discrepancy theory. We 
also generalize the method of pessimistic estimators due to Raghavan (J. Computer and System Sciences, 
. , 37 (1988), pp. I3O-I43), to obtain constructive (algorithmic) versions of our results for covering integer 

' programs. 
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^ 1 Introduction 

The powerful Lovasz Local Lemma (LLL) is often used to show the existence of rare combinatorial struc- 
^ ^ tures by showing that a random sample from a suitable sample space produces them with positive probabil- 
d • ity ^3]; see Alon & Spencer and Motwani & Raghavan [271 for several such applications. We present an 
extension of this lemma, and demonstrate applications to rounding fractional solutions for certain families 
of integer programs. 

Let e denote the base of natural logarithms as usual. The symmetric case of the LLL shows that all of a 
set of "bad" events Ei can be avoided under some conditions: 

Lemma 1.1. ([14]) Let Ei, E2, ■ ■ ■ , Em be any events with Pr(£'j) < p Vi. // each Ei is mutually indepen- 
dent of all but at most d of the other events Ej and if ep{d -|- 1) < 1, then Pr(Ai^i ^i) > 0- 

*A preliminary version of this work appeared as a paper of the same title in the Proc. ACM-SIAM Symposium on Discrete 
Algorithms, pages 6-15, 1996. Work done in parts at: (i) the University of Maryland (supported in part by NSF Award CCR- 
0208005), (ii) the National University of Singapore, (iii) DIMACS (supported in part by NSF-STC91-19999 and by support 
from the N.J. Commission on Science and Technology), (iv) the Institute for Advanced Study, Princeton, NJ (supported in 
part by grant 93-6-6 of the Alfred P. Sloan Foundation), and (v) while visiting the Max-Planck-Institut fiir Informatik, 66123 
Saarbriicken, Germany. 

^Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, 
MD 20742, USA. E-mail: srin@cs.umd.edu. 
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Though the LLL is powerful, one problem is that the "dependency" d is high in some cases, precluding 
the use of the LLL if p is not small enough. We present a partial solution to this via an extension of the 
LLL (Theorem lij.ip . which shows how to essentially reduce d for a class of events Ei] this works well when 
each Ei denotes some random variable deviating "much" from its mean. In a nutshell, we show that such 
events Ei can often be decomposed suitably into sub-events; although the sub-events may have a large 
dependency among themselves, we show that it suffices to have a small "bipartite dependency" between 
the set of events Ei and the set of sub-events. This, in combination with some other ideas, leads to the 
following applications in integer programming. 

It is well-known that a large number of NP-hard combinatorial optimization problems can be cast as integer 
linear programming problems (ILPs). Due to their NP-hardness, good approximation algorithms are of 
much interest for such problems. Recall that a p-approximation algorithm for a minimization problem is 
a polynomial-time algorithm that delivers a solution whose objective function value is at most p times 
optimal; p is usually called the approximation guarantee, approximation ratio, or performance guarantee 
of the algorithm. Algorithmic work in this area typically focuses on achieving the smallest possible p in 
polynomial time. One powerful paradigm here is to start with the linear programming (LP) relaxation of 
the given ILP wherein the variables are allowed to be reals within their integer ranges; once an optimal 
solution is found for the LP, the main issue is how to round it to a good feasible solution for the ILP. 
Rounding results in this context often have the following strong property: they present an integral solution 
of value at most y* ■ p, where y* will throughout denote the optimal solution value of the LP relaxation. 
Since the optimal solution value OPT of the ILP is easily seen to be lower-bounded by y*, such rounding 
algorithms are also ^-approximation algorithms. Furthermore, they provide an upper bound of p on the 
ratio OPT/y*, which is usually called the integrality gap or integrality ratio of the relaxation; the smaller 
this value, the better the relaxation. 

This work presents improved upper bounds on the integrality gap of the natural LP relaxation for two 
families of ILPs: minimax integer programs (MIPs) and covering integer programs (CIPs). (The precise 
definitions and results are presented in §l2j) For the latter, we also provide the corresponding polynomial- 
time rounding algorithms. Our main improvements are in the case where the coefficient matrix of the given 
ILP is column-sparse: i.e., the number of nonzero entries in every column is bounded by a given parameter 
a. There are classical rounding theorems for such column-sparse problems (e.g., Beck &: Fiala [0], Karp, 
Leighton, Rivest, Thompson, Vazirani & Vazirani 18 ). Our results complement, and are incomparable 
with, these results. Furthermore, the notion of column-sparsity, which denotes no variable occurring in 
"too many" constraints, occurs naturally in combinatorial optimization: e.g., routing using "short" paths, 
and problems on hypergraphs with "small" degree. These issues are discussed further in § |2l 

A key technique, randomized rounding of linear relaxations, was developed by Raghavan & Thompson |S2] 
to get approximation algorithms for such ILPs. We use Theorem 13. II to prove that this technique produces, 
with non-zero probability, much better feasible solutions than known before, if the constraint matrix of 
the given MIP/CIP is column-sparse. (In the case of MIPs, our algorithm iterates randomized rounding 
several times with different choices of parameters, in order to achieve our result.) Such results cannot be 
got via Lemma ll.ll as the dependency d, in the sense of Lemma ll.ll can be as high as 0(m) for these 
problems. Roughly speaking. Theorem 13. II helps show that if no column in our given ILP has more than 
a nonzero entries, then the dependency can essentially be brought down to a polynomial in o; this is the 
key driver behind our improvements. 

Theorem 13.11 works well in combination with an idea that has blossomed in the areas of derandomization 
and pseudorandomness, in the last two decades: (approximately) decomposing a function of several vari- 
ables into a sum of terms, each of which depends on only a few of these variables. Concretely, suppose 
Z is a sum of random variables Zi. Many tools have been developed to upper-bound Pr(Z — E[Z] > z) 
and Pr(|Z — E[Z]| > z) even if the ZiS are only (almost) A;- wise independent for some "small" k, rather 
than completely independent. The idea is to bound the probabilities by considering E[(Z — E[Z])''] or 



2 



similar expectations, which look at the Zi k or fewer at a time (via linearity of expectation). The main 
application of this has been that the Zi can then be sampled using "few" random bits, yielding a deran- 
domization/pseudorandomness result (e.g., [3 1131 El UHl I2H1 ES! ) • Our results show that such ideas can in 
fact be used to show that some structures exist! This is one of our main contributions. 

What about polynomial-time algorithms for our existential results? Typical applications of Lemma ll.ll are 
"nonconstructive" [i.e., do not directly imply (randomized) polynomial-time algorithmic versions], since 
the positive probability guaranteed by Lemma 11.11 can be exponentially small in the size of the input. 
However, certain algorithmic versions of the LLL have been developed starting with the seminal work of 
Beck These ideas do not seem to apply to our extension of the LLL, and hence our MIP result is 
nonconstructive. Following the preliminary version of this work |35j . two main algorithmic versions related 
to our work have been obtained: (i) for a subclass of the MIPs 20 , and (ii) for a somewhat different notion 
of approximation than the one we study, for certain families of MIPs JTj . 

Our main algorithmic contribution is for CIPs and multi-criteria versions thereof: we show, by a general- 
ization of the method of pessimistic estimators |3J , that we can efficiently construct the same structure 
as is guaranteed by our nonconstructive argument. We view this as interesting for two reasons. First, the 
generalized pessimistic estimator argument requires a quite delicate analysis, which we expect to be useful 
in other applications of developing constructive versions of existential arguments. Second, except for some 
of the algorithmic versions of the LLL developed in |24| I25j , most current algorithmic versions minimally 
require something like "pd'^ = 0(1)" (see, e.g., [SIP); the LLL only needs that pd = 0(1). While this issue 
does not matter much in many applications, it crucially does, in some others. A good example of this is the 
existentially-optimal integrality gap for the edge-disjoint paths problem with "short" paths, shown using 
the LLL in j2j. The above-seen "pd'^ = 0(1)" requirement of currently- known algorithmic approaches to 
the LLL, leads to algorithms that will violate the edge-disjointness condition when applied in this context: 
specifically, they may route up to three paths on some edges of the graph. See for a different - random- 
walk based - approach to low-congestion routing. An algorithmic version of this edge-disjoint paths result 
of |21j is still lacking. It is a very interesting open question whether there is an algorithmic version of the 
LLL that can construct the same structures as guaranteed to exist by the LLL. In particular, can one of 
the most successful derandomization tools - the method of conditional probabilities or its generalization, 
the pessimistic estimators method - be applied, fixing the underlying random choices of the probabilistic 
argument one-by-one? This intriguing question is open (and seems difficult) for now. As a step in this 
direction, we are able to show how such approaches can indeed be developed, in the context of CIPs. 

Thus, our main contributions are as follows, (a) The LLL extension is of independent interest: it helps in 
certain settings where the "dependency" among the "bad" events is too high for the LLL to be directly 
applicable. We expect to see further applications/extensions of such ideas, (b) This work shows that certain 
classes of column-sparse ILPs have much better solutions than known before; such problems abound in 
practice (e.g., short paths are often desired/required in routing), (c) Our generalized method of pessimistic 
estimators should prove fruitful in other contexts also; it is a step toward complete algorithmic versions of 
the LLL. 

The rest of this paper is organized as follows. Our results are first presented in § |2l along with a discussion 
of related work. The extended LLL, and some large-deviation methods that will be seen to work well with 
it, are shown in §121 Sections and El are devoted to our rounding applications. Finally, § El concludes. 

2 Our Results and Related Work 

Let Zj^ denote the set of non- negative integers; for any k £ [k] = {1, . . . , A;}. "Random variable" is 
abbreviated by "r.v.", and logarithms are to the base 2 unless specified otherwise. 

Definition 2.1. (Minimax Integer Programs) An MIP (minimax integer program) has variables W 
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and {xij : i E [n],j £ [ii]}, for some integers {ii}- Let N = ii and let x denote the N -dimensional 

vector of the variables Xij (arranged in any fixed order). An MIP seeks to minimize W , an unconstrained 
real, subject to: 

(i) Equality constraints: Vi G [n] X]je[^ ] "^^.i ~ 

(ii) a system of linear inequalities Ax < W, where A G [0,1]™^^ and W is the m-dimensional vector 
with the variable W in each component, and 

(Hi) Integrality constraints: Xi j £ {0,1} Vi,j. 

We let g denote the maximum column sum in any column of A, and a be the maximum number of non-zero 
entries in any column of A. 

To see what problems MIPs model, note, from constraints (i) and (iii) of MIPs, that for all i, any feasible 
solution will make the set {xj.j : j G [£i]} have precisely one 1, with all other elements being 0; MIPs 
thus model many "choice" scenarios. Consider, e.g., global routing in VLSI gate arrays j32j . Given are an 
undirected graph G = {V,E), a function p : V V, and Vi G a set Pi of paths in G, each connecting 
i to p{i); we must connect each i with p{i) using exactly one path from Pi, so that the maximum number 
of times that any edge in G is used for, is minimized-an MIP formulation is obvious, with Xij being the 
indicator variable for picking the jth path in Pj. This problem, the vector-selection problem of ^2]) and 
the discrepancy-type problems of Section |1J are all modeled by MIPs; many MIP instances, e.g., global 
routing, are NP-hard. 

We now introduce the next family of integer programs that we will work with. 

Definition 2.2. (Covering Integer Programs) Given A G [0,1]*"^"", b G [l,oo)™ and c G [0,1]" with 
maxj Cj = 1, a covering integer program (CIP) seeks to minimize c'^ ■ x subject to x £ and Ax > b. If 
A G {0,1}™^", each entry of b is assumed integral. We define B = miuj bi, and let a be the maximum 
number of non-zero entries in any column of A. A CIP is called unweighted if Cj = 1 Vj, and weighted 
otherwise. 

Note the parameters g, a and B of definitions 12.11 and 12.21 Though there are usually no restrictions on 
the entries of A, b and c in CIPs aside of non-negativity, it is well-known and easy to check that the above 
restrictions are without loss of generality. CIPs again model many NP-hard problems in combinatorial 
optimization. Recall that a hypergraph H = {y,E) is a family of subsets E (edges) of a set V (vertices). 
The classical set cover problem-covering V using the smallest number of edges in E (and its natural 
weighted version) is a standard example of a CIP. The parameter a here is the maximum number of 
vertices in any edge. 

Next, there is growing interest in multi- criteria optimization, since different participating individuals 
and/or organizations may have different objective functions in a given problem instance; see, e.g., |29j . 
Motivated by this, we study multi-criteria optimization in the setting of covering problems: 

Definition 2.3. (Multi- criteria CIPs; informal) A multi- criteria CIP has a system of constraints 
"Ax > b" as in CIPs, and has several given non-negative vectors ci, C2, . . . , q; the aim is to keep all 
the values cf ■ x "low". (For instance, we may aim to minimize maxj • x subject to Ax > b.) As in 
Definition \2.'A we assume that A G [0, 1]™^"^ ft £ [1, oo)"^ and for all i, Ci G [0, 1]" with maxj Cij = 1. 

We now present a lemma to quantify our approximation results; its proof is given in ^ 

Lemma 2.4. Given independent r.v.s Xi, . . . , Xn G [0, 1], let X = Y17=i -^i ^'^'^ ~ ^i-^]- 

a. For any 6 > 0, Pi{X > //(I + 6)) < G(//, 6), where G(/x, <5) = (eV(l + 6)^+^)''. 
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b. V/i > Vp G (0, 1), 3d = H{fj,,p) > such that \n6~\ • 6) < p and such that 

Given an ILP, we can find an optimal solution x* to its LP relaxation efficiently, but need to round 
fractional entries in x* to integers. The idea of randomized rounding is: given a real v > 0, round v to 
[v\ + 1 with probability v— [v\, and round v to [v\ with probability 1 — u + [t;J . This has the nice property 
that the mean outcome is v. Starting with this idea, the analysis of 32.^ produces an integral solution of 
value at most y* + 0(min{y*,m} • H (mm{y* , m} , 1/m)) for MIPs (though phrased a bit differently); this 
is derandomized in [SJ- But this does not exploit the sparsity of A; the previously-mentioned result of 
[TH] produces an integral solution of value at most y* + g + 1. 

For CIPs, the idea is to solve the LP relaxation, scale up the components of x* suitably, and then perform 
randomized rounding; see Section [3 for the details. Starting with this idea, the work of leads to certain 
approximation bounds; similar bounds are achieved through different means by Plotkin, Shmoys & Tardos 
j3L)j . Work of this author improved upon these results by observing a "correlation" property of CIPs, 
getting an approximation ratio of 1 + 0(ma,x{ln{mB /y*) / B , y^ln(mB /y*)/ B}). Thus, while the work of 
j32| gives a general approximation bound for MIPs, the result of |18j gives good results for sparse MIPs. 
For CIPs, the current-best results are those of IHE]; however, no better results were known for sparse CIPs. 

2.1 Improvements achieved 

For MIPs, we use the extended LLL and an idea of Eva Tardos that leads to a bootstrapping of the LLL 
extension, to show the existence of an integral solution of value y* + 0(min{y*, m} • H (inm{y* , m} , 1/a)) -|- 
0(1); see Theorem l4.5l Since a < m, this is always as good as the y*-l-0(min{y*, m} ■ H {inm{y* , m} , 1/m)) 
bound of [S2] and is a good improvement, if o <C m. It also is an improvement over the additive g factor 
of in cases where g is not small compared to y*. 

Consider, e.g., the global routing problem and its MIP formulation, sketched above; m here is the number 
of edges in G, and g = a is the maximum length of any path in |J- Pj. To focus on a specific interesting 
case, suppose y*, the fractional congestion, is at most one. Then while the previous results ([52] and 
[THj . resp.) give bounds of 0(logm/loglogm) and 0{a) on an integral solution, we get the improved 
bound of O (log a/ log log a). Similar improvements are easily seen for other ranges of y* also; e.g., if 
y* = O(logo), an integral solution of value O(loga) exists, improving on the previously known bounds of 
0(logm/log(21ogm/loga)) and 0(a). Thus, routing along short paths (this is the notion of sparsity for 
the global routing problem) is very beneficial in keeping the congestion low. Section 0] presents a scenario 
where we get such improvements, for discrepancy-type problems In particular, we generalize a 

hypergraph-partitioning result of Fiiredi & Kahn [16| . 

Recall the bounds of IHEI for CIPs mentioned in the paragraph preceding this subsection; our bounds 
for CIPs depend only on the set of constraints Ax > b, i.e., they hold for any non-negative objective- 
function vector c. Our improvements over |36j get better as y* decreases. We show an integrality gap 
of 1 -|- 0(max{ln(o -|- 1)/B, Y^ln(a -|- 1)/B}), once again improving on for weighted CIPs. This CIP 
bound is better than that of j36j if y* < mB/a: this inequality fails for unweighted CIPs and is generally 
true for weighted CIPs, since y* can get arbitrarily small in the latter case. In particular, we generalize 
the result of Chvatal ^U] on weighted set cover. Consider, e.g., a facility location problem on a directed 
graph G = (V, A): given a cost q E [0, 1] for each i £ V, we want a min-cost assignment of facilities to the 
nodes such that each node sees at least B facilities in its out-neighborhood-multiple facilities at a node 
are allowed. If Aj„ is the maximum in-degree of G, we show an integrality gap of 1 -|- 0(max{ln(Aj„ -|- 
1)/B, y/ln(S(Ai„ + l))/B}). This improves on [MI if y* < \V\B/A,n; it shows an 0(1) (resp., 1 + o(l)) 
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integrality gap if B grows as fast as (resp., strictly faster than) log Aj^. Theorem 15 . 71 presents our covering 
results. 

A key corollary of our results is that for families of instances of CIPs, we get a good (0(1) or 1 + o(l)) 
integrality gap if B grows at least as fast as log a. Bounds on the result of a greedy algorithm for CIPs 
relative to the optimal integral solution, are known jl2l I13j . Our bound improves that of ^2] and is 
incomparable with for any given A, c, and the unit vector 6/||6||2, our bound improves on ^S] if B 
is more than a certain threshold. As it stands, randomized rounding produces such improved solutions for 
several CIPs only with a very low, sometimes exponentially small, probability. Thus, it does not imply a 
randomized algorithm, often. To this end, we generalize Raghavan's method of pessimistic estimators to 
derive an algorithmic (polynomial-time) version of our results for CIPs, in § 15.31 

We also show via Theorem 15.91 and Corollarv 15. Ifll that multi-criteria CIPs can be approximated well. In 
particular, Corollarv l5.1Ul shows some interesting cases where the approximation guarantee for multi-criteria 
CIPs grows in a very much sub-linear fashion with the number i of given vectors q: the approximation 
ratio is at most 0(loglog£) times what we show for CIPs (which correspond to the case where i = 1). We 
are not aware of any such earlier work on multi-criteria CIPs. 

The preliminary version of this work was presented in • As mentioned in § two main algorithmic 
versions related to our work have been obtained following First, for a subclass of the MIPs where 

the nonzero entries of the matrix A are "reasonably large" , constructive versions of our results have been 
obtained in [20]. Second, for a notion of approximation that is different from the one we study, algorithmic 
results have been developed for certain families of MIPs in Furthermore, our Theorem 15.71 for CIPs 
has been used in :19; to develop approximation algorithms for CIPs that have given upper bounds on the 
variables xj. 

3 The Extended LLL and an Approach to Large Deviations 

We now present our LLL extension. Theorem 13.11 For any event E, define x(-^) to be its indicator r.v.: 
1 if £' holds and otherwise. Suppose we have "bad" events Ei, . . . , Em with a "dependency" d' (in 
the sense of Lemma II. 1() that is "large" . Theorem 13.11 shows how to essentially replace d' by a possibly 
much-smaller d, under some conditions. It generalizes Lemma ll. II (define one r.v., Ci^i = xi^i), for each 
i, to get Lemma ll.l() . its proof is very similar to the classical proof of Lemma ll.ll and its motivation will 
be clarified by the applications. 

Theorem 3.1. Given events Ei, . . . , Em and any I C [m], let Z[I) = /\-^jEi. Suppose that for some 
positive integer d, we can define, for each i G [m], a finite number of r.v.s Ci^i, Ci^2, ■ ■ ■ each taking on only 
non-negative values such that: 

(i) any dj is mutually independent of all but at most d of the events E^, k ^ i, and 

(ll) yi C (H - {i}), Pv{E, I Z{I)) < E[a,j I Z{I)]. 

Let Pi denote E[Cij]; clearly, Pr(£'j) < pi (set I = (p in (ii))- Suppose that for all i G [m] we have 
epi{d+l) < 1. Then Pr(Ai^) > (d/(d + 1))™ > 0. 

Remark 3.2. Cij and Cij' can "depend" on different subsets of {Ek\k ^ i}; the only restriction is that 
these subsets be of size at most d. Note that we have essentially reduced the dependency among the EiS, to 
just d: epi{d + 1) < 1 suffices. Another important point is that the dependency among the r.v.s Cij could 
be much higher than d: all we count is the number of E^ that any Cij depends on. 
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Proof of Theoreni AS. li We prove by induction on |/| that if i / then Pr(-E'j | Z{I)) < epi, which suffices 
to prove the theorem since Pr(/\-£'j) = niG[m](-'- ~ I -^([^ ~ I])))- the base case where 1 = 0, 

Pr(£'j I Z{I)) = Pr(£'i) < pi. For the inductive step, let Sijj = {k £ I \ Cij depends on Ef^}, and 
S'i j J = I — Sijj; note that l-Sij,/] < d. If Sijj = 0, then E[Cij | Z{I)] = E[Cij]. Otherwise, letting 
= {^1, ■ ■ ■ ,^r}, we have 

• X(^(g.,,,/)) I ^(^U/)] E[C,, I Z(g^,)] 

^ ' ^ Pr(^(5,,,/) I - Pr(Z(5,,-,) I 

since Cjj- is non-negative. The numerator of the last term is E[Cjj], by assumption. The denominator can 
be lower-bounded as follows: 

n (1 - Pr(^£. I Zi{h,e2, . . . , 4-1} U 5^ ,))) > n (1 - ^^'^^) ^ (1 - + 1))' > {d/id + 1))" > 1/e; 
se[r] s6H 

the first inequality follows from the induction hypothesis. Hence, E[Cjj' | Z{I)] < eE[Cij] and thus, 
Pr(^i I Z{I)) < Zj I Z{I)] < ep, < l/{d +1). ' ' □ 

The crucial point is that the events Ei could have a large dependency d', in the sense of the classical 
Lemma ll. II The main utility of Theorem l3.1l is that if we can "decompose" each Ei into the r.v.s Qj- that 
satisfy the conditions of the theorem, then there is the possibility of effectively reducing the dependency 
by much {d' can be replaced by the value d). Concrete instances of this will be studied in later sections. 

The tools behind our MIP application are our new LLL, and a result of j33j . Define, for z = (zi, . . . , Zn) £ 
3?", a family of polynomials Sj{z),j = 0, 1, . . . , n, where So{z) = 1, and for j G [n], 

^ji^)= Zi^Zi^---Zi.. (1) 

l< ti <i2---<ij <n 

Remark 3.3. For real x and non-negative integral r, we define (^) = x{x — 1) • • • (x — r + l)/r! as usual; 
this is the sense meant in Theorem \3.4\ below. 

We define a nonempty event to be any event with a nonzero probability of occurrence. The relevant 
theorem of j^S] is the following: 

Theorem 3.4. ([33j) Given r.v.s Xi, . . . ,X„ e [0, 1], let X = J27=i Xi and = E[X]. Then, 

(a) For any q > 0, any nonempty event Z and any non-negative integer k < q, 

Pr(X >q\Z)< E[Yk,, \ Z], 

where Yk,g = Sk{Xi,...,Xn)/ it). 

(h) If the XiS are independent, (5 > 0, and k = Ifid], then Pr(X > ^(1 + 6)) < E[Yf.^^(^i_^_s)] < G{n,6), 
where G(-, •) is as in Lemma\ 



(c) If the XiS are independent, then E[Sk{Xi, . . . < (^) • (/i/n)''' < /k\. 

Proof. Suppose ri,r2,...r„ G [0,1] satisfy X^iLi — ^- Then, a simple proof is given in for the 

fact that for any non-negative integer k < q, Sk{ri,r2, . . . ,rn) > (^). This clearly holds even given the 
occurrence of any nonempty event Z. Thus we get Pr(X > q) \ Z) < VilY^^q >^\Z)< 'Ei\Y}^^q \ Z\, where 
the second inequality follows from Markov's inequality. The proofs of (b) and (c) are given in [SSj. □ 
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We next present the proof of Lemma \2Al 

Proof of Lemma \2.4\ Part (a) is the Chernoff-Hoeffding bound (see, e.g., Appendix A of or [27\). For 
(b), we proceed as follows. For any ^ > 0, it is easy to check that 

G(/.,<5) = e-®(^^') if 5 G (0,1); (2) 
G(/x,(5) = e-e{M(i+<5)in{i+5)) iis>l. (3) 



Now if /i < log{p ^)/2, choose 



C 



log(p 



/ilog(log(p 

for a suitably large constant C. Note that 6 is lower-bounded by some positive constant; hence, © holds 
(since the constant 1 in the conditions "(5 G (0, 1)" and "5 > 1" of ((3) and © can clearly be replaced by 
any other positive constant). Simple algebraic manipulation now shows that if C is large enough, then 

IfJ-SI •G{fi, 6) < p holds. Similarly, if > log(p^^)/2, we set6 = C- for a large enough constant 

C, and use 0. □ 



4 Approximating Minimax Integer Programs 

Suppose we are given an MIP conforming to Definition 12.11 Define t to be maxj^^^j A^Zj, where NZi is 
the number of rows of A which have a non-zero coefficient corresponding to at least one variable among 
{xij : j € [ii]}. Note that 

g < a < t < min{m, a ■ maxjg (4) 

Theorem 14.21 now shows how Theorem 13.11 can help, for sparse MlPs-those where t <^ m. We will then 
bootstrap Theorem l4.2l to get the further improved Theorem l4.51 We start with a proposition, whose proof 
is a simple calculus exercise: 

Proposition 4.1. If < fii < IJ-2, then for any 5 > 0, G{^i, / ^^i) < G{fi2,S)- 

Theorem 4.2. Given an MIP conforming to Definition \2.1l randomized rounding produces a feasible 
solution of value at most y* + [min{y*,m} • H {mm{y* , m} , l/(et))], with non-zero probability. 

Proof. Conduct randomized rounding: independently for each i, randomly round exactly one Xij to 1, 
guided by the "probabilities" {x*j}- We may assume that {x* j} is a basic feasible solution to the LP 
relaxation. Hence, at most m of the {x*^} will be neither zero nor one, and only these variables will 
participate in the rounding. Thus, since all the entries of A are in [0, 1], we assume without loss of generality 
from now on that y* < m (and that max^gj^j^j < m); this explains the "min{?/*,m}" term in our stated 
bounds. If z G {0, 1}^ denotes the randomly rounded vector, then E[(74z)j] = bi by linearity of expectation, 
i.e., at most y*. Defining k = \y*H{y*, l/(et))] and events Ei,E2, Em hy Ei = ''{Az)i > bi + fc", we 
now show that Pr(Aje[m] ^i) ^ ^ using Theorem 13.11 Rewrite the ith constraint of the MIP as 

Xi^r < W, where Xi^r = ^ Ai^(^r,s)Xr,s] 
r€[n] s£[£r] 

the notation ^i,(r,s) assumes that the pairs {(r, s) : r G [n],s G [ir]} have been mapped bijectively to [A^], 
in some fixed way. Defining the r.v. 
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we note that for each i, the r.v.s {Zi^r '■ r £ [n]} he in [0, 1] and are independent. Also, Ei = "X]re[n] ^i,r ^ 
bi + k\ 

Theorem 13.41 suggests a suitable choice for the crucial r.v.s Cij (to apply Theorem 13. Ij) . Let u = (^); we 
now define the r.v.s {Cij : i G [iTi],j G [u]} as follows. Fix any i £ [m]. Identify each j £ [u] with some 
distinct A:-element subset S{j) of [n], and let 

We now need to show that the r.v.s Cij satisfy the conditions of Theorem 13.11 For any i S [m], let 
6i = k/bi. Since hi < y*, we have, for each i £ [m], 

G{bi,6i) < G(y*,fc/y*) (by Proposition mU 

< Giy*,Hiy*,l/{et))) 

< \/{ekt) (by the definition of H). 

Now by Theorem 13.41 we get 

Fact 4.3. For all i G [m] and for all nonempty events Z , Pv{Ei \ Z) < J2je[u] I ^^so, pi = 

j:,elu]E[Ci,i] < G{h,6i) < l/{ekt). 

Next since any dj involves (a product of) k terms, each of which "depends" on at most (t — 1) of the 
events {E^ : v £ {[m] — {i})} by definition of t, we see the important 

Fact 4.4. Vi £ [m] Vj £ [u], Cij £ [0, 1] and Cij "depends" on at most d = k{t — 1) of the set of events 
{E,:v£{[m]-{i})}. 

From Facts and ETH and by noting that epi{d + 1) < e{kt — k + \)/{ekt) < 1, we invoke Theorem 13.11 
to see that Pr(/\^gj^] Ei) > 0, concluding the proof of Theorem 14.21 □ 

Theorem 14.21 gives good results if t <C m, but can we improve it further, say by replacing t hy a {< t) in 
it? As seen from the key reason for t ^> a®^^^ is that maxjg[„] li ^ a®^^^. If we can essentially "bring 
down" maxjg[„] li by forcing many x* ^ to be zero for each i, then we effectively reduce t {t < a ■ maxi £i, 
see (@}); this is so since only those x* j that are neither zero nor one take part in the rounding. A way of 
bootstrapping Theorem 14.21 to achieve this is shown by: 

Theorem 4.5. For any given MIP, there exists an integral solution of value at most y*+0(l)+0(min{y*, m}- 
H (mm{y* , m} , 1/a)). 

Proof. Let Kq > be a sufficiently large absolute constant. Now if 

{y* > t^^^) or {t < max{Ko, 2}) or {t < a^) (6) 

holds, then we will be done by Theorem 14.21 So we may assume that © is false. Also, if y* < 
Theorem 14.21 guarantees an integral solution of value 0(1); thus, we also suppose that y* > t-^/'^. The 
basic idea now is, as sketched above, to set many x* j to zero for each i (without losing too much on y*), 
so that maxj £i and hence, t, will essentially get reduced. Such an approach, whose performance will be 
validated by arguments similar to those of Theorem l4.21 is repeatedly applied until © holds, owing to the 
(continually reduced) t becoming small enough to satisfy There are two cases: 

Case I: y* > 1. Solve the LP relaxation, and set x'-j := {y*)'^{log^ t)x* j. Conduct randomized rounding 
on the x[ j no w, rounding each x[j independently to Zij £ {[x-^J, [a^^j]}- (Note the key difference from 
Theorem 14.21 where for each i, we round exactly one x^j to 1.) 
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Let Ki > be a sufficiently large absolute constant. We now use ideas similar to those used in our proof 
of Theorem 14.21 to show that with nonzero probability, we have both of the following: 

ViG[m], (^z)i < (y*)3log5t.(l + Ki/((y*)i-5log2t)), and (7) 

Vi G [n], I ^ z,,j - {y*f log^ t\ < K,y* log^ t. (8) 
j 

To show this, we proceed as follows. Let Ei, E2, ■ ■ ■ , Em be the "bad" events, one for each event in ^ not 
holding; similarly, let E^+i, Em+2, ■ ■ ■ 1 Em+n be the "bad" events, one for each event in (jH)) not holding. 
We want to use our extended LLL to show that with positive probability, all these bad events can be 
avoided; specifically, we need a way of decomposing each Ei into a finite number of non-negative r.v.s Cij. 
For each event E^+e where i > 1, we define just one r.v. Cm+eX- ^^^^ indicator variable for the 

occurrence of Em+i- For the events Ei where i < m, we decompose Ei into r.v.s Cjj just as in ©: each 
such Cij is now a scalar multiple of at most 

0{{y*f log' t/{{y*)'-' log2 t)) = 0{{y*)'-' log' t) = 0{t^-''" log' t) 

independent binary r.v.s that underlie our randomized rounding; the second equality (big-Oh bound) here 
follows since © has been assumed to not hold. Thus, it is easy to see that for all i, 1 < i < m + n, and 
for any j, the r.v. Cjj depends on at most 

0{t ■t^-'/'' log' t) (9) 

events E^, where k ^ i. Also, as in our proof of Theorem 14.21 Theorem 13.41 gives a direct proof of 
requirement (ii) of Theorem 13.11 part (b) of Theorem 13.41 shows that for any desired constant K, we can 
choose the constant Ki large enough so that for all i, - E[Cjj] < . Thus, in view of ©, we see by 
Theorem rm that Pr(AIlt"^) > 0. 

Fix a rounding z satisfying and ©• For each i e [n] and j G [^j], we renormalize as follows: x'l^ := 
Zijl Yliu ^i,u- Thus we have J2u ^'lu ~ ^ ^^'^ ^' "^^^ ihaA, we have two very useful properties. First, 
since Zjj > (y*)^ log^ t ■ — 0(^*Y~^)) fo^' * from Q, we have. Mi £ [m], 

M.",. s '^''v s .-(1 + o(i/(.. log^ m. (10) 

1 - 0{l/{y* logH)) 

Second, since the Zj.j are non-negative integers summing to at most {y*flog't{l + 0{l/{y* log^ t))), at 
most 0((y*)^ log^ t) values x'- j are nonzero, for each i E [n]. Thus, by losing a little in y* (see (|1()|)). our 
"scaling up-rounding-scaling down" method has given a fractional solution x" with a much-reduced ii for 
each i; £i is now 0((y*)^ log^ t), essentially. Thus, t has been reduced to 0{a{y*)'^ log' t); i.e., t has been 
reduced to at most 

K2t'/^+^/'log't (11) 

for some constant K2 > that is independent of Kq, since (jHI was assumed false. Repeating this 
scheme O(loglogt) times makes t small enough to satisfy ©• More formally, define to = and ^i+i = 

sequence at the first point where either t = ti satisfies ©, or tj+i > ti 
holds. Thus, we finally have t small enough to satisfy © or to be bounded by some absolute constant. 
How much has maxjg[^] (Ax)j increased in the process? By ([Til]), we see that at the end, 

max(Ax)i < y* ■ "Tf (1 + 0(l/(y* log^ t^))) < . e«(S.>o Vfe* log'*.)) < y* + 0{l), (12) 

since the values logtj decrease geometrically and are lower-bounded by some absolute positive constant. 
We may now apply Theorem 14.21 
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Case II: t^^/^ < y* < 1. The idea is the same here, with the scahng up of x*j being by (log^t)/y*; the 
same "scahng up-rounding~scahng down" method works out. Since the ideas are very similar to Case I, 
we only give a proof sketch here. We now scale up all the x* j first by {\og^ t)/y* and do a randomized 
rounding. The analogs of Q and Q that we now want are 

ViG[m], (Az)i < log^ t- {I + K[/log^t), and (13) 
Vi G [n], - log^ t/y*\ < K[ log^ t/^. (14) 

3 

Proceeding identically as in Case I, we can show that with positive probability, (|13() and (|14|) hold simul- 
taneously. Fix a rounding where these two properties hold, and renormalize as before: x'l j := zi^j/ Zi^u- 
Since (fT3|) and (|TI|) hold, it is easy to show that the following analogs of (fT?!]) and (fTT|) hold: 



(Ax-0.< f';r°;y'°<''' <.'(l + O(l/Vi)); and 

1 - 0{y/y^ /\ogH) 



t has been reduced to 0{alog^ t/y*), i.e., to 0{t^/'^+^/'^ log^ t). 
We thus only need O(loglogt) iterations, again. Also, the analog of ((T^ now is that 

max(^x)i < y* • 17(1 + 0(1/ log^tj)) < y* . e^(^^>o ^/^^e" <y* + 0{l). 

This completes the proof. □ 

We now study our improvements for discrepancy-type problems, which are an important class of MIPs 
that, among other things, are useful in devising divide-and-conquer algorithms. Given is a set-system 
{X,F), where X = [n] and F = {Di, D2, ■ ■ ■ , Dm} C 2-^. Given a positive integer £, the problem is to 
partition X into i parts, so that each Dj is "split well": we want a function f : X ^ [i] which minimizes 
maXjg[jvf],fcG[£] |{^ £ Dj : f{i) = k}\. (The case £ = 2 is the standard set-discrepancy problem.) To 
motivate this problem, suppose we have a (di)graph {V, A); we want a partition of V into Vi, . . . ,Ve such 
that yv £ V, e N{v)r\Vk}\ : k G [£]} are "roughly the same", where N{v) is the (out-)neighborhood of 
V. See, e.g., |21E1 for how this helps construct divide-and-conquer approaches. This problem is naturally 
modeled by the above set-system problem. 

Let A be the degree of {X,F), i.e., maxjg[„]|{j : i £ Dj}\, and let A' = maxjj.^p \Dj\. Our problem 
is naturally written as an MIP with m = M£, £i = £ for each i, and (7 = a = A, in the notation of 
Definition 12. H y* = A'/£ here. The analysis of 32 gives an integral solution of value at most y*{\ + 
0{H{y* ,1/{M £)))), while ^Hl presents a solution of value at most y* + A. Also, since any Dj G F 
intersects at most (A — 1)A' other elements of F, Lemma 1 1 . 1 1 shows that randomized rounding produces, 
with positive probability, a solution of value at most y*{l + 0{H{y*, l/(eA'A^)))). This is the approach 
taken by for their case of interest: A = A', ^ = A/ log A. 

Theorem 14 . 51 shows the existence of an integral solution of value y*{l + 0{H{y* , 1/A))) + 0(1), i.e., removes 
the dependence on A'. This is an improvement on all the three results above. As a specific interesting 
case, suppose £ grows at most as fast as A'/ log A. Then we see that good integral solutions-those that 
grow at the rate of 0{y*) or better-exist, and this was not known before. (The approach of ^HI shows 
such a result for £ = 0(A'/ log(max{A, A'})). Our bound of 0(A'/logA) is always better than this, and 
especially so if A' » A.) 
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5 Approximating Covering Integer Programs 

One of the main ideas behind Theorem 13.11 was to extend the basic inductive proof behind the LLL by 
decomposing the "bad" events Ei appropriately into the r.v.s Cij. We now use this general idea in a 
different context, that of (multi-criteria) covering integer programs, with an additional crucial ingredient 
being a useful correlation inequality, the FKG inequality ^H] • The reader is asked to recall the discussion of 
(multi-criteria) CIPs from § [2 We start with a discussion of randomized rounding for CIPs, the Chernoff 
lower-tail bound, and the FKG inequality in § 15.11 These lead to our improved, but nonconstructive, 
approximation bound for column-sparse (multi-criteria) CIPs, in § 15.21 This is then made constructive in 
§ 15.31 we also discuss there what we view as novel about this constructive approach. 

5.1 Preliminaries 

Let us start with a simple and well-known approach to tail bounds. Suppose y is a random variable and 
y is some value. Then, for any < (5 < 1, we have 



where the inequality is a consequence of Markov's inequality. 

We next setup some basic notions related to approximation algorithms for (multi-criteria) CIPs. Recall 
that in such problems, we have £ given non-negative vectors ci, C2, . . . , Q such that for all i, Cj G [0, 1]" with 
maxj Cij = 1; ^ = 1 in the case of CIPs. Let x = [x* *) denote a given fractional solution that 

satisfies the system of constraints Ax > b. We are not concerned here with how x* was found: typically, x* 
would be an optimal solution to the LP relaxation of the problem. (The LP relaxation is obvious if, e.g., 
£ = 1, or, say, if the given multi-criteria aims to minimize maxj cj ■ x*, or to keep each cf ■ x* bounded by 
some target value Vi.) We now consider how to round x* to some integral z so that: 

(PI) the constraints Az > b hold, and 

(P2) for all i, cJ ■ z is "not much bigger" than cJ ■ x*: our approximation bound will be a measure of how 
small a "not much bigger value" we can achieve in this sense. 

Let us now discuss the "standard" randomized rounding scheme for (multi-criteria) CIPs. We assume 
a fixed instance as well as x*, from now on. For an a > 1 to be chosen suitably, set x'j = cexj, for 
each j G [n]. We then construct a random integral solution z by setting, independently for each j G [n], 
Zj = [x'j\ + 1 with probability x'j — [x'j\, and zj = [x'j\ with probability 1 — {x'j — [x'j\). The aim then is 
to show that with positive (hopefully high) probability, (PI) and (P2) happen simultaneously. We now 
introduce some useful notation. For every j G [n], let Sj = L^jJ • Let Ai denote the ith row of A, and let 
Xi,X2, ■ ■ ■ ,Xn G {0, 1} be independent r.v.s with Fr(Xj = 1) = x'j — sj for all j. The bad event Ei that 
the ith constraint is violated by our randomized rounding is given by Ei = ^^Ai ■ X < ^j(l — (5i)", where 
fii = E[j4j • X] and 5i = 1 — {bi — Ai ■ s)/fii. We now bound Pr(£'j) for all i, when the standard randomized 
rounding is used. 



Pr(y <y)< Pr((l - Sy > (1 - 5)^) < 



(1 - 6)y 



(15) 



Lemma 5.1. Define g{B,a) = (a • e 



Then for alii, 



Pr(^,) < 




B{a-lf/{2a) 



under standard randomized rounding. 
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Proof. The first inequality follows from H15() . Next, the ChernofF-Hoeffding lower-tail approach [IJ 
shows that 

E[(l - ^ / e-'^ 

(1 - - Ul-f^i)^"^' 

It is observed in (and is not hard to see) that this latter quantity is maximized when sj = for all 
j, and when each bi equals its minimum value of B. Thus we see that Pr(i?j) < g(B,a). The inequality 
g{B,a) < e~-^("~-^) for a > 1, is well-known and easy to verify via elementary calculus. □ 

Next, the FKG inequality is a useful correlation inequality, a special case of which is as follows |15| . Given 
binary vectors a = (ai, 02, • • • , a^) G {0, 1}^ and b = {bi,b2, ■ ■ ■ ,bi) G {0, 1}^, let us partially order them by 
coordinate- wise domination: a ^ 6 iff < 6j for all i. Now suppose li, ^2, . . . , are independent r.v.s, 
each taking values in {0, 1}. Let Y denote the vector {Yi,Y2., ■ ■ ■ ,Y(). Suppose an event A is completely 
defined by the value of Y . Define A to be increasing iff: for all a G {0, 1}^ such that A holds when Y = a, A 
also holds when Y = b, for any b such that a <b. Analogously, event A is decreasing iff: for all a G {0, 1}^ 
such that A holds when Y = a., A also holds when Y = b, for any b < a. The FKG inequality proves 
certain intuitively appealing bounds: 

Lemma 5.2. (FKG inequality) Let Ii, I2, ■ ■ ■ , It be any sequence of increasing events and Di, D2, . . . ,Dt 

be any sequence of decreasing events (each Ii and Di completely determined by Y). Then for any i G \t\ 
and any S C \t\, 

(i) Pr(/,| A,e5^j) > and Pr(A| K.esD,) > Pr(A); 

(ii) Pr(/i| Aje5^j) < PrU*) and Pr(A| AjeS^i) < Pr(A). 

Returning to our random variables Xj and events Ei, we get the following lemma as an easy consequence 
of the FKG inequality, since each event of the form "Ei" or "Xj = 1" is an increasing event as a function 
of the vector {Xi,X2, ■ ■ ■ , Xn)'- 

Lemma 5.3. For all Bi,B2 ^ [m] such that Bir\B2 = and for any B^ C [n], Pr(AigBi I ((AjeB2 ^i)^ 

{K,^B^{Xk = l)))>W,^B,^T{Ei). 

5.2 Nonconstructive approximation bounds for (multi-criteria) CIPs 

Definition 5.4. (The function TZ) For any s and any ji < j2 < ••• < js, l^t I^iji, j2, ■ ■ ■ , js) be 

the set of indices i such that row i of the constraint system "Ax > b" has at least one of the variables 
jki ^ 1^ k < s, appearing with a nonzero coefficient. (Note from the definition of a in Defn. \2.'A that 
I^(ji,j2, • • • ,js)\ <a-s.) 

Let the vector x* = (x^, Xg, . . . , x* ), the parameter a > 1, and the "standard" randomized rounding 
scheme, be as defined in § 15.11 The standard rounding scheme is sufficient for our (nonconstructive) 
purposes now; we generalize this scheme as follows, for later use in S 15.31 

Definition 5.5. (General randomized rounding) Given a vector p = {pi,P2, ■ ■ ■ ,Pn) G [0,1]", the 
general randomized rounding with parameter p generates independent random variables Xi , X2 , ■ ■ ■ , Xn G 
{0, 1} with Fr(Xj = 1) = pj; the rounded vector z is defined by Zj = [ax* J + Xj for all j. (As in the 
standard rounding, we set each zj to be either [ax*\ or \aXj~\ ; the standard rounding is the special case in 
which E[zj] = ax* for all j.) 

We now present an important lemma, Lemma 15.61 to get correlation inequalities which "point" in the 
"direction" opposite to FKG. Some ideas from the proof of Lemma ll.ll will play a crucial role in our proof 
this lemma. 
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Lemma 5.6. Suppose we employ general randomized rounding with some parameter p, and that Pr(Ai!Li ^i) 
is nonzero under this rounding. The following hold for any q and any 1 < ji < j2 < " " " < jq ^ n. 

(^) 

ft{x,, = x,, = ... = x,^ = 1 1 a ^) < n a vHE)v 

i=l llie7^(ji,i2,...,i,)V-^--^^VAjj 

the events Ei = {{Az)i < hi) are defined here w.r.t. the general randomized rounding, 
(a) In the special case of standard randomized rounding, 

n (l-Pr(£;.))>(l-5(B,a)r; (17) 

i<^'RUl,j2,-,jq) 

the function g is as defined in Lemma 15. li 

Proof, (i) Note first that if we wanted a lower bound on the l.h.s., the FKG inequality would immediately 
imply that the l.h.s. is at least PjiPj2 ' " Pjq- We get around this "correlation problem" as follows. Let 
Q = T^Ui,j2, ■ ■ -Jq); let Q' = [m] - Q. Let 

Z^ = {/\E'i), andZ2 = {/\E;). 



Letting Y = n?=i ^jt ) "^oie that 
Now, 

Pr(y = 1 I (Zi A Z2)) 



IQI < ag and (18) 
Y is independent of Z2. (19) 

Pr(((y = 1) A Zi) I Z2) 

Pr(Zi I Z2) 
Pr((y = 1) I Z2) 

Pr(Zi I Z2) 
Pr(y = 1 



< 



PrfZi I Z2) 



(by m) 



^ n ""'"^'^"^l "Prfi^)) (byLemmaESl. 

(ii) We get from Lemma lO and (fTH)) . □ 

We will use Lemmas 15.31 and 15.61 to prove Theorem 15.91 As a warmup, let us start with a result for the 
special case of CIPs; recall that y* denotes • x*. 

Theorem 5.7. For any given CIP, suppose we choose q,/3 > 1 such that I3{1 — g{B,a)Y > 1- Then, there 
exists a feasible solution of value at most y*a(i. In particular, there is an absolute constant K > such 
that if a, f5 > 1 are chosen as: 

a = K ■ln{a + l)/B and p = 2, if ln{a + l)> B, and (20) 
a = P=l + K- Vln(a+ 1)/S, if ln(a + 1) <B; (21) 

then, there exists a feasible solution of value at most y*a.(3. Thus, the integrality gap is at most 1 + 
0{max{ln{a + 1)/B, ^ln(o + 1)/S}). 
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Proof. Conduct standard randomized rounding, and let £ be the event that ■ z > y*a(5. Setting 
Z = Aig[m] ^'^'i ^^ ^ E[c^ • z] = y*a, we see by Markov's inequality that Pr(<S | Z) is at most 
R = (X]j=i P'^l^i — 1 I Note that Pr(Z) > since a > 1; so, we now seek to make i? < 1, 

which will complete the proof. Lemma 15.61 shows that 



fi(i-{l-g{B,a)Y fi{l-9{B,a)Y' 
thus, the condition (5{1 — g{B,a))"' > 1 suffices. 

Simple algebra shows that choosing a,f3> 1 as in (PU]) and (PT|). ensures that P{1 — g{B,a))^ > 1. □ 

The basic approach of our proof of Theorem l5.7l is to follow the main idea of Theorem 13. 11 and to decompose 
the event | Z" into a non- negative linear combination of events of the form "Xj = 1 | Z'; we then 
exploited the fact that each Xj depends on at most a of the events comprising Z. We now extend 
Theorem 15 . 71 and also generalize to multi-criteria CIPs. Instead of employing just a "first moment method" 
(Markov's inequality) as in the proof of Theorem 15. 71 we will work with higher moments: the functions 5^ 
defined in (Q) and used in Theorem 13.41 Suppose some parameters Aj > are given, and that our goal is 
to round x* to z so that the event 

A = '\Az >b) A (Vi, cf • z < Xi)" (22) 

holds. We first give a sufficient condition for this to hold, in Theorem 15.91 we then derive some concrete 
consequences in Corollarv l5.1Ul We need one further definition before presenting Theorem 15. 91 Recall that 
Ai and bi respectively denote the ith row of A and the ith component of b. Also, the vector s and values 
6i will throughout be as in the definition of standard randomized rounding. 

Definition 5.8. (The functions ch and cW ) Suppose we conduct general randomized rounding with some 
parameter p; i.e., let Xi,X2, . . . ,X„ be independent binary random variables such that Vt^Xj = 1) = pj. 
For each i £ [m], define 

(Note from I115\) that if we conduct general randomized rounding with parameter p, then Pr((^2;)j < bi) < 
chj(p) < ch.[{p); also, "ch" stands for "ChernojJ-HoejJding".) 

Theorem 5.9. Suppose we are given a multi- criteria CIP, as well as some parameters Ai, A2, . . . , A^ > 0. 
Let A be as in \2'J\) . Then, for any sequence of positive integers (ki,k2, . . . ,k£) such that ki < Xi, the 
following hold. 

(i) Suppose we employ general randomized rounding with parameter p = {pi,P2, ■ ■ ■ ,Pn)- Then, Pr(^) is 
at least 

m = ( n (1 - ^^r(p)) ) - E ■ E f n ^^.^-^ • ■ n (1 - ^^rip))-^ (23) 

\-reH / i=l UJ ji<--<jk^ \t=i / rm{h,-,hi) 

as in Defn. Cij £ [0, 1] is the jth coordinate of the vector Ci. 

(a) Suppose we employ the standard randomized rounding to get a rounded vector z. Let Xi = z/j(l + 'ji) 
for each i £ [£], where Vi = E[c^ ■ z] = a - {cj ■ x*) and 7i > is some parameter. Then, 

^{p) > (1 - g(B, a)T • 1 - E ' " " ^^^^ 

\ i=l \ ki ) / 

In particular, if the r.h.s. of \24\ ) is positive, then Pr(^) > for standard randomized rounding. 
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The proof is a simple generalization of that of Theorem 15 .71 and is deferred to Section [5.41 Theorem 15 .71 is 
the special case of Theorem 15.91 corresponding to i = ki = 1. To make the general result of Theorem 15.91 
more concrete, we now study an additional special case. We present this special case as one possible "proof 
of concept", rather than as an optimized one; e.g., the constant "3" in the bound "c?" • z < 3fj" can be 
improved. 

Corollary 5.10. There is an absolute constant K' > such that the following holds. Suppose we are given 
a multi- criteria CIP with notation as in part (ii) of Theorem \5.yi Define a = K' ■ max{ ^°("')+^ in(2£) ^ -^-^ 
Now if Ui > log^(2£) for all i G [i], then standard randomized rounding produces a feasible solution z such 
that cf ■ z < 3vi for all i, with positive probability. 

In particular, this can be shown by setting ki = [ln(2^)] and 7j = 2 for all i, in part (ii) of Theorem \5.iA 

Proof. Let us employ Theorem I5.9r ii) with ki = \ln{2£)~\ and 7i = 2 for all i. We just need to establish 
that the r.h.s. of (|24j) is positive. We need to show that 

i=l \ ki ) 

it is sufficient to prove that for all i, 

(1 - g{B, a))-'^-'^' < 1/1 (25) 



We make two observations now. 



• Since /cj ~ ln£ and Ui > log (2£), 

• (1 — g{B, a))^"''^^ can be made arbitrarily close to 1 by choosing the constant K' large enough. 
These two observations establish (I25p . □ 



5.3 Constructive version 

It can be shown that for many problems, randomized rounding produces the solutions shown to exist by 
Theorem 15.71 and Theorem 15.91 with very low probability: e.g., probability almost exponentially small in 
the input size. Thus we need to obtain constructive versions of these theorems. Our method will be a 
deterministic procedure that makes 0{n) calls to the function $(•), in addition to poly(n,m) work. Now, 
if k' denotes the maximum of all the ki, we see that $ can be evaluated in poly(n'^ ,m) time. Thus, our 
overall procedure runs in time poly(n*^ , m) time. In particular, we get constructive versions of Theorem 15. 71 
and Corollarv 15.101 that run in time poly(n, m) and poly(n^°^^, m), respectively. 

Our approach is as follows. We start with a vector p that corresponds to standard randomized rounding, 
for which we know (say, as argued in Corollary I5.1U() that ^{p) > 0. In general, we have a vector of 
probabilities p = {pi,P2, ■ ■ ■ ,Pn) such that <I>(p) > 0. If p G {0, l}**, we are done. Otherwise suppose some 
Pj lies in (0, 1); by renaming the variables, we will assume without loss of generality that j = re. Define 
p' = {pi,P2, ■ ■ ■ ,Pn-i, 0) and p" = {pi,P2, ■ ■ ■ ,Pn-i, 1)- The main fact we wish to show is that $(p') > 
or ^(p") > 0: we can then set pn to or 1 appropriately, and continue. (As mentioned in the previous 
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paragraph, we thus have 0{n) calls to the function <!>(•) in total.) Note that although some of the pj will lie 
in {0, 1}, we can crucially continue to view the Xj as independent random variables with Fr{Xj = 1) = pj. 

So, our main goal is: assuming that pn S (0, 1) and that 

Hp) > 0, (26) 

to show that ^(p') > or <I>(p") > 0. In order to do so, we make some observations and introduce some 
simplifying notation. Define, for each i S [m]: qi = ch^(p), q[ = ch-(p'), and q'l = ch.[{p"). Also define the 
vectors q = (q'l, (?2, • • ■ , Qm)-, l' = Wi^lk^ - ■ ■ ^ I'm)^ q" = {q'l, q'-^-, ■ ■ ■ , q'm)- We now present a useful lemma 
about these vectors: 



Lemma 5.11. For all i G [m], we have 



< gf<g^<l; (27) 
qi > Pnq'l + - PrMi, and (28) 
q'l = qiifi^n{n). (29) 



Proof. The proofs of ()27|) and H29|) are straightforward. As for (|28() . we proceed as in [SH]- First of all, if 
= 1, then we are done, since g-', q[<l. So suppose qi < this case, qi = chj(p). Now, Definition 15.81 
shows that 

chi(p) = pnch.i{p") + (1 - pn)dii{p'). 
Therefore, qt = chi(p) = pnch.i{p") + (1 - Pn)ch.i{p') > p„ch-(/') + (1 - p„)ch-(y). □ 

Since we are mainly concerned with the vectors p, p' and p" now, we will view the values pi,P2, ■ ■ ■ ,Pn-i 
as arbitrary but fixed, subject to (|26|) . The function $(•) now has a simple form; to see this, we first define, 
for a vector r = (ri, r2, . . . , r^) and a set {/ C [m], 

/(C/,r) = n(l-^.)- 

ieu 

Recall that pi,p2, ■ ■ ■ ,Pn-i are considered as constants now. Then, it is evident from (|^^ that there exist 
constants ui,U2, ■ ■ ■ ,ut and f i, f2, . . . , Uf, as well as subsets Ui, U2, ■ ■ ■ ,Ut and Vi, V2, ■ ■ ■ ,Vti of [m], such 
that 

>i • /(C/i,g)) - (p„ -yZ^j- fiVj,q)); (30) 



$(p) 


= f{[m\,q)- 


$(p') 


= f{[m],q')- 


Hp") 


= f{[m],q")- 




= fi[m],q")- 



^u.-f{U.,q"))-{Y,vrf{yj,Q"))- (32) 

i j 

Importantly, we also have the following: 

the constants Ui,Vj are non-negative; Vj, Vj n Tl{n) = 0. (33) 

Recall that our goal is to show that ^{p') > or ^{p") > 0. We will do so by proving that 

^'(p) < Pn^'(y') + (1 - Pn)Hp')- (34) 
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Let us use the equalities (jSOI), (EU), and (jSH). In view of ^ and (jSSl), the term "-p„ • X^^- vj ■ f{Vj,qY 
on both sides of the inequahty (|5H) cancels; defining A{U) = (1 — Pn) • f{U,q') + Pn ■ fiU,q") — f{U,q), 
inequality (|34|) reduces to 

A{[m])-Y,u^■A{Ui)>0. (35) 

i 

Before proving this, we pause to note a challenge we face. Suppose we only had to show that, say, A([m]) 
is non-negative; this is exactly the issue faced in j36j . Then, we will immediately be done by part (i) of 
Lemma l5. 121 which states that A([/) > for any set U. However, (|35|) also has terms such as "ui • A(C/i)" 
with a negative sign in front. To deal with this, we need something more than just that A(C/) > for all 
U ; we handle this by part (ii) of Lemma l5.12l We view this as the main novelty in our constructive version 
here. 

Lemma 5.12. Suppose U <Z V Q [m]. Then, (i) A{U) > 0, and (ii) A{U)/f{U,q) < A{V)/f{V,q). (Since 
> by \2f^) . we have that qi<l for each i. So, l/f{U,q) and l/f{V,q) are well-defined.) 

Assuming that Lemma 15.121 is true, we will now show (|35() : the proof of Lemma 15.121 is given below. We 
have 

A([m]) -Y^u,- A{Ui) = (A(H)//(H, g)) • /(H, g) - ^(A(C/,)//(f/„ q)) ■ ■ f{U,, q) 



> (A(H)//(H,g)) 

> (by (EHI) and (jSnj) 



f{[m],q) -^Ui- f{Ui,q) 



(by Lemma l5.12|) 



Thus we have (jSSJ. 



Proof of Lemma 15.121 It suffices to show the following. Assume U ^ [m]; suppose u £ ([m] — U) and 
that U' = U U {u}. Assuming by induction on \U\ that A{U) > 0, we show that A{U') > 0, and that 
A{U)/ f{U, q) < A{U')/ f{U' , q). It is easy to check that this way, we will prove both claims of the lemma. 

The base case of the induction is that \U\ G {0, 1}, where A{U) > is directly seen by using ((^HJ. Suppose 
inductively that A{U) > 0. Using the definition of A{U) and the fact that f{U',q) = (1 — qu)f{U,q), we 
have 

f{U',q) = {l-q,).[{l-p,,)f{U,q')+Pnf{U,q")-A{U)] 

< (1 - (1 - Pn)qu - Pnq'L) ■ [(1 - Pn)f{U, q') + Pnf{U, q")] - (1 - qu) ■ A{U), 

where this last inequality is a consequence of H28() . Therefore, using the definition of A(U') and the facts 
f{U',q') = (1 - q'Jf{U,q') and /([/', g") = (1 - q'!,)f{U,q"), 

A{U') = {l-p^){l-q'Jf{U,q')+Pnil-q':)fiU,q")-f{U',q) 

> (1 - pn){l - q'u)fiU, q') + pn{l - q'L)f{U, q") + 

(1 - qu) ■ A{U) - (1 - (1 - Pn)q'u - PuQu) ■ [(1 - Pn)f{U, q') + p„/([/, q")] 
= (1 - qu) ■ A{U)+Pn{l-Pn) ■ {f{U,q") - f{U,q')) • (g^ - q^) 

> il-qu)-A{U) (by (EH)). 

So, since we assumed that A{U) > 0, we get A{U') > 0; furthermore, we get that A{U') > (1 — qu) ■ A{U), 
which implies that A{U')/ f{U' ,q) > A{U)/f{U,q). □ 
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5.4 Proof of Theorem 15.91 



(i) Let Er = {(Az)r < br) be defined w.r.t. general randomized rounding with parameter p; as observed 
in Definition 15.81 Pr(£',.) < chj,(p). Now if ch^(p) = 1 for some r, then part (i) is trivially true; so we 
assume that Pr(£'r) < ch^(p) < 1 for all r G [m]. Defining Z = (Az > b) = Are[m] ^r, we get by the FKG 
inequality that 

Pr(Z) > n (1 - 

r€:[m] 

Define, for i = 1,2, . . . the "bad" event £i = {cj • z > Aj). Fix any i. Our plan is to show that 



Pv{£i I Z) < 



1 



fXA I iiCi,it -Pjt I • 

ji<j2<-<jk, \t=i / \reniji,j2,-;jki) 



n (l-Pr(i^.))-i 



(36) 



If we prove H36|) . then we will be done as follows. We have 



Pr(^) > Pr(Z) • ( 1 - E I ^) J > ( n (1 - • ( 1 - E I ^) ) 



(37) 



Now, the term "(HreHCl ~ P^(-^r')))" is a decreasing function of each of the values Pr(£',.); so is the 
lower bound on "— Pr(£'j | Z)" obtained from Hence, bounds and H37() . along with the bound 

Pr(£'r) < ch^(p), will complete the proof of part (i). 

We now prove (|36j) using Theorem 13. 4f a) and Lemma 15.61 Recall the symmetric polynomials Sk from (^. 
Define Y = 5fe^(Q,iXi, ^,2^2, . . . , Q,„X„)/(^;) . By Theorem Ulta) , Pr(fi | Z) < B[Y \ Z\. Next, the 
typical term in E[y | Z] can be upper bounded using Lemma 15.61 



E 



< 



n 



_ t=i i=i 
Thus we have , and the proof of part (i) is complete 
(ii) We have 



Y[t=l ^i,jt ' Pjt 



Hp) 



U^^-^Kip)) 



r£ m 



i^-izi^- E p^.] ■ ( n 1 _ eh- („) 



(38) 



Lemma 123 shows that under standard randomized rounding, ch^(p) < g{B,a) < 1 for all r. So, the r.h.s. 
K of (|38|) gets lower-bounded as follows: 

\ 1=1 [ k, ) ji<-<jk, \t=i / renUu-,jk,) J 

\ i=i \ k, ) ji<-<jk^ \t=i / / 



where the last line follows from Theorem 13. 4r c). 



□ 
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6 Conclusion 

We have presented an extension of the LLL that basicahy helps reduce the "dependency" much in some 
settings; we have seen apphcations to two famihes of integer programming problems. It would be interesting 
to see how far these ideas can be pushed further. Two other open problems suggested by this work are: (i) 
developing a constructive version of our result for MIPs, and (ii) developing a poly(n, m)-time constructive 
version of Theorem 15.91 as opposed to the poly(n'^ ,m)-time constructive version that we present in S 15.31 
Finally, a very interesting question is to develop a theory of applications of the LLL that can be made 
constructive with (essentially) no loss. 
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